Adaptive Fuzzy Expert System for Sign Recognition

The Hand Motion Understanding (HMU) system is a vision-based Australian sign language recognition system that recognises static and dynamic hand signs. It uses a visual hand tracker to extract 3D hand configuration data from a visual motion sequence, and a classifier that recognises the changes of these 3D kinematic data as a sign. This paper presents the HMU classifier that uses an adaptive fuzzy inference engine for sign recognition. Fuzzy set theory allows the system to express the sign knowledge in natural and imprecise descriptions. The HMU classifier has an adaptive engine that trains the system to be adaptive to the errors caused by the tracker or the motion variations exhibited amongst the signers. The HMU system is evaluated with 22 static and dynamic Auslan signs, and recognised 20 signs before training, and 21 signs after training of the HMU classifier.